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In this paper, we present findings from an exploratory study of mathematics education 
stakeholders to understand their professional networks, and acquisition and use of research on 
mathematics teaching and learning. Evidence suggests that mathematics leaders are key to 
promoting organizational sensemaking and are more likely to acquire and use research on 
mathematics teaching and learning which has important implications for improvement efforts at 
scale. 
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Introduction 

Historic approaches to bridging the research-practice divide have often focused on improving 
the quality of research dissemination efforts to move evidence from research to use in practice. 
These approaches privileged researchers’ perspectives, and though they achieved some success, 
the field lacks empirical understanding of what forms of research evidence are being drawn upon 
and used in practice (Finnigan, Daly, & Che, 2012). To improve the use of research in practice, 
scholars have identified a number of resources and characteristics of schools and districts, such 
as the influence of mid-level decision makers in organizations because they often “straddle 
policy and practice and are well poised to put research to work” (Tseng, 2012, p.5). Others 
challenge the dissemination model and argue for other ways to relate the work of researchers and 
practitioners, such as research-practice partnerships (RPPs). RPPs have recently gained traction 
as a promising approach for educational improvement (Coburn & Penuel, 2016). In RPPs, 
researchers and practitioners identify and commit to addressing a shared problem through long- 
term, mutualistic collaborations that include research, development, or evaluation (Coburn, 
Penuel, & Geil, 2013). 

As part of an RPP with our state education agency focused on improving the process of 
implementing new mathematics standards, we conducted an exploratory study of the professional 
networks and research uses of mathematics teachers, mathematics leaders (e.g. school-based 
coaches and curriculum facilitators, district-based math leaders), and school-based administrators 
(e.g. principals, assistant principals). This paper reports results from a questionnaire that was 
developed to inform the design of professional learning opportunities for mathematics teachers 
and leaders that centralize research evidence on mathematics teaching and learning. In doing so, 
we aim and address the following research questions: (1) From whom and for what purposes do 
mathematics teachers, mathematics leaders, and administrators have significant conversations 
about mathematics teaching and learning?; (2) From what sources and to what extent do 
mathematics teachers, mathematics leaders, and administrators look for and use research on 
mathematics teaching and learning?; and (3) What do mathematics teachers, mathematics 
leaders, and administrators identify as primary barriers to using research in their role? 
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Background 

Our partnership began after our state adopted new mathematics standards. State agency 
leaders were interested in using an improvement science approach in build and refine a process 
of implementing state academic standards. During the first year of the partnership, we negotiated 
and specified research evidence and its use in practice among mathematics teachers and leaders 
as the focus of our work together and co-designed a variety of professional learning opportunities 
and materials for mathematics teachers and leaders that embody research on student learning, 
instructional practice, and teacher learning in professional development (Wilson, McCulloch, 
Webb, Stephan, Mawhinney, & Curtis, 2017). 

To inform the design of these efforts, we developed an exploratory questionnaire to 
understand the ways mathematics education stakeholders acquire and use evidence from 
mathematics education research in their practice. In what follows, we briefly summarize the 
literature on professional networks and research use, outline our theoretical perspective, and 
describe the development and administration of the exploratory questionnaire. We then highlight 
key findings and conclude with implications for others interested in promoting both 
organizational learning and the use of research in practice with attention to scale. 


Professional Networks and Research Use 

Scholars studying the ways research evidence informs practice and policy define research use 
as the act of drawing on and interacting with research evidence in the course of decision making 
(Coburn & Turner, 2011; Honig et al., 2017; Tseng, 2012). Investigations of practitioner and 
policy maker uses of research have identified several broad ways that evidence is used, for 
example, instrumental use which results in changes in practice (Nutley et al., 2007), conceptual 
use which results in changes in knowledge (Weiss et al., 1977), or imposed use in which 
practitioners are pressured to use research by agencies or policy makers (Weiss et al., 2005), 
among others. In addition, Honig and colleagues (2017) reported that though practitioners claim 
to use research in their work, it is often interpreted through existing schema to reinforce previous 
decisions or rejected ideas that conflict with prior understandings. 

Much of the scholarship on research use has focused on local or statewide data, such as 
student achievement scores or local measures. However, little attention has been paid to the ways 
mathematics education stakeholders use research evidence on mathematics teaching, students’ 
mathematical learning, and/or mathematics teacher learning to inform their decisions. For the 
purpose of this exploratory study, we wanted to focus on similarities and differences in the ways 
teachers and leaders acquire and use research evidence on students’ mathematical learning, 
mathematics teaching practices, and mathematics teacher learning given their direct relation to 
the work of teachers and coaches in the classroom. 

Studies on research use in practice often focus on singular roles (e.g. administrators, central 
office leaders), downplaying the social ecology and complex nature of research use processes 
within school and district initiatives. Relationships, organizational structures and contexts, and 
policy all complicate and influence the ways practitioners use research (Tseng, 2012). In 
addition, acquiring and using research evidence for improvement is a “multilevel phenomena” 
and occurs both within formal organizational structures and in informal social interactions (Daly 
et al., 2014). Scholars working in this area have called for more attention to developing an 
understanding of the ways research is used in practice within district systems between leaders 
and teachers (Honig & Venkateswaran, 2012) and how research supports implementation and 
improvement efforts (Daly & Finnigan, 2010). Some have found that small changes at different 
levels of the system can add up to larger organizational improvements (Coburn & Turner, 2012). 
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Research-practice partnerships are an increasingly popular approach to improve the use of 
research in practice (Coburn & Penuel, 2016). Proponents suggest partnerships support both 
sensemaking and use of research evidence in practice by creating opportunities for mutual 
engagement with research, translating research into tools that may serve as vehicles for learning 
and improvements to practice, and serving as models of the use of research (Fishman et al., 
2013). However, research has shown that intermediary organizations are insufficient absent 
leaders who continue to learn about research use while also teaching others (Honig, 2017). 
Moreover, research has highlighted the importance of “opinion leaders” positions in social 
networks (Palinkas et al., 2011) as conduits for both the acquisition and use of research in 
practice (Tseng, 2012). Thus, for this exploratory survey, we were interested not only in the 
types of mathematics education research used, but also in the reasons mathematics teachers, 
mathematics leaders, and administrators interact with one another and the ways these interactions 
may relate to the acquisition and use of research. 


Theoretical Perspectives 

Our partnership uses design-based implementation research (DBIR) (Fishman et al., 2013) to 
organize our development, implementation, and research efforts. DBIR focuses not only on 
developing and refining tools and environments for learning but also on creating structures and 
supports necessary to scale and sustain them (Fishman et al., 2013). Throughout implementation, 
partners seek to improve the design of implementation efforts, generate theories of learning and 
implementation, and create supporting infrastructures to develop capacity and sustainability. 
Specifically, we use Lave and Wenger’s (1991) ideas of participation, practice, and boundaries 
as a frame for designing for teacher and leader learning. We use organizational sensemaking 
(Weick, 1995) to broadly frame standards implementation as an organizational learning problem, 
and draw upon two sets of constructs related to the processes and kinds of resources that 
individuals or collectives within larger systems use when experiencing ambiguity or violated 
expectations during periods of systemic change. 

Our design process is guided by a set of design and implementation principles derived from 
our commitments to supporting mathematics teaching and learning, theories of learning and 
implementation, the research on teacher learning and professional development, and the diverse 
expertise of our partners (Wilson et al., 2017). In this paper, we attend to our principals of 
utilizing research on mathematics teaching, connecting teachers and leaders, and designing 
coordinated tools and resources to better understand mathematics teachers, leaders, and 
administrators professional networks and the ways they acquire and use research evidence on 
mathematics teaching and learning. 


Methods 

Questionnaire Design 

In the spring of 2017, we developed a questionnaire to inform ongoing efforts to co-design 
statewide initiatives related to standards implementation and the promotion of equitable learning 
opportunities for mathematics education stakeholders. We began by asking a set of demographic 
questions related to respondents’ school district, role, years of experience, and grade-band. We 
then focused on three constructs identified by the partnership as important to the design of 
learning opportunities for mathematics teachers and leaders: professional networks, use of 
research in practice, and instructional vision. In this paper, we focus our efforts on statewide 
responses from mathematics teachers, mathematics leaders (e.g. school- and district-based 
mathematics coaches, curriculum facilitators), and school administrators (e.g. principals, 
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assistant principals) related to professional networks and research use to inform our design. 

To answer our first research question focused on professional networks, we asked 
respondents two questions: with whom and how often respondents had “significant exchanges 
about mathematics teaching and learning in the past year”; and what the primary reason was for 
the majority of these exchanges. To answer our second research question focused on research 
use, we asked respondents questions about their acquisition and use of research specific to 
mathematics education (e.g. research on students’ mathematical thinking, mathematics teaching 
practices, mathematics teacher learning). These questions asked respondents to indicate how 
often they looked for research on topics related to mathematics teaching, the social sources of the 
research they encountered, and how likely they were to use research. To answer our third 
research question focused on barriers in using research, we asked respondents to select two main 
barriers to using research from eight possible choices. 

Several of the questions were adapted from a validated and reliable survey developed by The 
National Center for Research in Policy and Practice (ncrpp.org). The survey’s purpose was to 
characterize how leaders perceive, acquire, and use research to inform their decision-making. To 
reduce fatigue effect bias, we selected a subset of the survey questions most directly related to 
our research questions and implementation efforts and modified portions of these questions to 
reflect our focus. In addition, there were a set of questions focused on respondent demographics 
and another set of questions to directly inform future co-design and implementation efforts. Prior 
to administering the survey, we field tested the items with members of the partnership. We 
modified and clarified individual items as needed to ensure each measured the constructs of 
interest. 

Data and Analysis 

The questionnaire was distributed through listservs at the state agency to mathematics 
teachers, administrators, and mathematics leaders. The agency estimated the total number of 
educators on the listservs to be approximately 20,000. The questionnaire was open for 17 days 
during late May and June, 2016. Potential respondents received one message to invite 
participation upon its opening and three follow-up reminders throughout the open period. 

Responses from those who completed 80% of the items and gave consent for research were 
considered complete and as data. Our quantitative analysis proceeded in two stages. First, we 
used descriptive statistics and graphical displays to explore responses to individual questions and 
then groups of questions. We then used responses to demographic items to organize a search for 
relationships among professional networks and research use. Because the questionnaire was 
developed with a primary goal of informing our design efforts, findings are descriptive. 
Findings 

In total, 1,605 teachers, 197 leaders, and 104 administrators (N=1,906) responses were 
collected, with a response rate estimated to be approximately 20%. While low, the responses 
represented 100% of the eight state education regions and 114 of the 115 school districts in the 
state with a mean and median of 17 and 11 responses per school district respectively. To better 
understand the representation of respondents across our state, a ratio of students per respondent 
was calculated by taking the ratio of the total number of students in a district per number of 
respondents in a district. These scores were then averaged across all districts in each of the eight 
regions across the state to obtain an average ratio per region. The mean and median of these 
region averages were 948 and 916 students per respondent respectively (std. dev.=167) — 
indicating a reasonably acceptable distribution of respondents per region across the state. In what 
follows, we share findings organized by our research questions. 
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From whom and for what purposes do mathematics teachers, mathematics leaders, and 
administrators have significant conversations about mathematics teaching and learning? 


Respondents were asked to self-report how often they had significant conversations about 
mathematics teaching and learning in the last year by choosing none, 1-5 times, 6-10 times, or 
greater than 10 times. Results were translated to a scale of 0-3 (0 [none] — 3 [>10]) and means 
for each role are shown in Table 1. Results indicate that all roles self-report that they regularly 
engage in conversations about mathematics teaching and learning with teachers in schools. 
Mathematics leaders self-report that they more regularly engage with teachers and administrators 
about mathematics teaching and learning than do teachers or administrators, suggesting that 
leaders span boundaries within and across schools within districts. 


Table 1: Mean scores for likelihood of having significant exchanges about mathematics 


Teachers Math Leaders Administrators 
Teachers within schools 2.45 2.58 2.25 
Teachers across district 1.27 1.89 1.05 
Math Leaders 1.31 1.97 1.59 
Administrators 1.55 2.03 1.77 
Higher Ed Faculty 0.81 0.89 0.80 


Taken that mathematics leaders self-reported the regularity of their conversations with others, 
as well as teachers and administrators self-reports of their conversations with math leaders, Table 
2 highlights the predominant focus of math leaders’ conversations. Results indicate that math 
leaders have frequent conversations about mathematics teaching and learning across many 
domains of mathematics education. With teachers, these conversations focus on planning for 
instruction, instructional practices to support student learning that align with mathematics 
standards, and resources that can be used in instruction. With one another, these conversations 
focus on professional development and other activities to support teacher learning that focus on 
mathematics standards and instructional practices. With administrators, these conversations 
focus on curriculum, assessments, and instructional practices. Taken together, the frequency and 
focus of these exchanges may indicate that mathematics leaders are centrally connected to 
district-wide implementation efforts related to mathematics teaching and learning. 


Table 2: Percentage of respondents indicating the predominant focus of their exchanges. 
Math Leaders conversations with...were focused on... 
Teachers in schools Planning (35%), Teaching Practices (17%), Resources (15%) 
Teachers across district | Standards (22%), Planning/PD (17%), Teaching Practices (16%) 
Math Leaders Teaching (20%), Standards (17%), PD (15%) 
Administrators Curriculum (21%), Assessment (18%), Teaching Practices (16%), 


From what sources and to what extent do mathematics teachers, mathematics leaders, and 
administrators look for and use research on mathematics teaching and learning? 


Drawing from our design and implementation principle focused on utilizing tools based on 
research on mathematics teaching and learning, we sought to explore stakeholders’ frequency 


Hodges, T.E., Roy, G. J., & Tyminski, A. M. (Eds.). (2018). Proceedings of the 40th annual meeting of 
the North American Chapter of the International Group for the Psychology of Mathematics 
Education. Greenville, SC: University of South Carolina & Clemson University. 


Articles published in the Proceedings are copyrighted by the authors. 


Inservice Teacher Education/Professional Development 347 


and use of research in practice. Respondents were asked questions to indicate how frequently 
they (1) looked for research in the past year and (2) previously used research in their role on a set 
of mathematics teaching and learning related topics. Results were translated to a scale of 0-3 (0 
[never], 1 [rarely], 2 [sometimes], 3 [often]) and means across respondents for teachers (n=677), 
mathematics leaders (n=81), administrators (n=36) are shown in Table 3. For the survey, each 
respondent was given the set of questions related to professional networks and then randomly 
assigned questions related to either research use or instructional vision, thus the number of 
respondents for research use and instructional vision are roughly half of the total number of 
respondents. 


Table 3: Frequency of prior use of research and looking for research in the past year 


How frequently have stakeholders Teachers Math Leaders | Administrators 
looked for & used research on Looked | Prior | Looked | Prior | Looked | Prior 
math teaching and learning? For Use For Use For Use 
Students’ mathematical thinking 1.7 2.0 2.0 25 1.8 23 
Mathematics teaching practices 1.8 aed 2.3 pee) 2.0 25 
Math professional development 1.4 1.8 2.1 Ze 1.6 22 
Resources for instruction 2.1 ae 2.4 22 20) 2.4 
Assessment practices 1.8 2.0 Zl 2.3 1 2.4 


Results indicate that all roles self-report that they look for and using research on mathematics 
teaching and learning. In addition, mathematics leaders self-report data suggest that they both 
look for and use research more often than teachers and administrators across each domain of 
research on mathematics teaching and learning. Moreover, data indicate that respondents are 
more likely to use research than they are to look for it, which suggests that they may acquire 
research from a variety of sources. Respondents were then asked to indicate how likely they were 
to acquire research from a list of social resources. Results were translated to a scale of 0-3 (0 
[never], 1 [rarely], 2 [sometimes], 3 [often]) and means for each role are shown in Table 4. 


Table 4: Mean scores likelihood of acquiring research from the following sources 


Teachers Math Leaders Administrators 
Teachers within schools 2.1 1.7 2.3 
Teachers across district 1.5 1.7 1.7 
Math Leaders 1.4 1.9 2.0 
Administrators 1.3 1.2 2.1 
Higher Ed Faculty 0.6 1,2 0.4 
Professional Associations 1.1 1.9 1.4 
Research or Practice Journals 0.8 1.9 1.2 


Results indicate that all roles self-report that they rarely to sometimes acquire research from 
the list of sources. For teachers, they reported that they were most likely to acquire research from 
other teachers. For administrators, they reported that they were most likely to acquire research 
from other administrators or mathematics leaders. For mathematics leaders, they reported that 
they were most likely to acquire research from other math leaders, professional associations, or 
journals. In addition, if one assumes that higher education faculty have access to or are 
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producing research on mathematics teaching and learning, mathematics leaders reported that they 
were more likely to acquire research from faculty than teachers or administrators. 


What do mathematics teachers, mathematics leaders, and administrators identify as the primary 
barriers to using research in their role? 


Drawing from our design and implementation principles and our commitment to build 
capacity for sustainability at scale, we sought to explore what stakeholders’ identified as the 
primary barriers to using research in their role. Respondents were asked to choose two main 
barriers from a list of hypothesized barriers we gathered through our conversations with 
educators. Results were tabulated across respondents for each role as shown in Table 5. 


Table 5: Percentages of primary identified barriers to using research in practice 


Teachers Leaders Administrators 
Time constraints 37% 32% 33% 
Lack of PD that shares research 21% 22% 20% 
Cost/Lack of access to journals 9% 17% 20% 
Lack of relevance or ease of use 19% 11% 10% 
Timeliness of research (e.g. out of date) 8% 8% 13% 
Not valued in my education community 3% 7% 2% 
Lack of high-quality evidence 5% 3% 3% 


Results indicate that the primary barrier to using research in practice across all roles was time 
constraints. Secondly, results indicate that teachers, leaders, and administrators all reported a 
lack of access to professional development focused on sharing research findings as a barrier. 


Implications and Discussion 

The purpose of this exploratory study was to inform our ongoing efforts to co-design 
professional learning opportunities for mathematics teachers and leaders. Results of our analysis 
suggest that mathematics leaders are key aspects of school and district communication 
infrastructures related to mathematics teaching and learning. An examination of responses about 
their use of research provides some evidence that this communication shares research findings 
with others, on occasion. Moreover, leaders are more likely to seek out, promote, and use 
research on mathematics teaching and learning. 

Results from this questionnaire will inform our upcoming efforts to co-design learning 
opportunities for leaders and teachers focused on mathematics teaching and learning that embody 
both research and attention to scale. For researchers and practitioners engaged in large scale 
improvement initiatives, results from our descriptive questionnaire mirror emerging findings 
about the influences of mid-level decision makers in decisions and the role of professional 
networks in introducing new ideas to schools and districts. We also note that self-reports of 
practice that are deemed favorable are regularly overestimated. While it is promising that leaders 
reported these practices, it is also noteworthy that teachers, leaders, and administrators were 
unlikely to have significant conversations about research on mathematics teaching and learning 
with higher education faculty. If we, as mathematics education researchers purport to both 
engage in and share these forms of research, it is important for us to consider these results. 

As we consider the conference theme, “Looking Back, Looking Ahead”, we see attention to 
mathematics leaders as one way to meet enduring challenges of shaping mathematics teaching. 
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Leveraging their professional networks and supporting them in learning to use research in 
practice may promote coherence and systemic change as they mitigate shifting policies, changes 
in curriculum, and expectations of industry. We see this work as supporting research-based 
mathematics teaching that leads to more equitable learning opportunities for students. 
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